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Abstract Body 


Background / Context: 

Children enter kindergarten with variable levels of mathematics skill and knowledge 
gained from infonnal learning opportunities at home, preschool, and daycare. Many perform 
well once they receive formal mathematics instruction. However, if students do not develop an 
initial understanding of the most basic aspects of formal mathematics, they are likely to have 
difficulty understanding or succeeding at mathematics they will encounter in first grade and 
beyond. This puts these students at an academic disadvantage that is likely to become 
exacerbated over time. In fact, longitudinal studies of mathematics achievement have suggested 
that kindergarten mathematics concepts are pivotal foundational skills for future mathematics 
performance (Duncan, et al., 2007; Morgan et ah, 2009). Accordingly, there has been a targeted 
effort to create effective early mathematics interventions that support the development of whole 
number understanding through robust number sense curricula (Clarke, et al., 2011; Jordan, 
Glutting, Dyson, Hassinger-Das, & Irwin, 2012). 

Early mathematics interventions are often intended for use in Response to Intervention 
(Rtl) systems to support the mathematics achievement of students at all levels (Fuchs, Fuchs, & 
Compton, 2012; Fuchs, Fuchs, & Hollenbeck, 2007) and to prevent students from developing 
mathematics learning difficulties. As Rtl approaches gain traction, there has been particular 
interest in the development of Tier 2 interventions that provide intensive instruction to students 
above and beyond their core mathematics instruction. However schools encounter barriers 
related to resources and time requirements for screening students, implementing interventions, 
and progress monitoring in Rtl systems. 

Technology-based interventions are a potentially promising means of surmounting those 
barriers. This is particularly true if the interventions customize instruction to meet individual 
students’ needs in a way not feasible in a classroom or resource-strapped context. In fact, 
technology holds great promise as an answer to the intervention implementation dilemma. 
Devices, software, and apps are often quite affordable and compatible with classroom contexts. 
However, questions remain as to whether or not educational technology can be considered a 
reliable means for improving student outcomes (Kebritchi et al., 2010; Slavin & Lake, 2008; 
Collins & Halverson, 2009; Poison & Morgan, 2010; Young, et al., 2012). 

In an effort to merge the promise of educational technology with the need for early 
mathematics interventions that can be utilized in Rtl systems, the study reported here details 
findings from the development and feasibility trials of a kindergarten iPad-based whole number 
intervention. The efforts reported here detail attempts to develop an effective kindergarten 
mathematics curriculum delivered on a hand-held tablet. Adherence to research-based 
curriculum development procedures (Clement, 2007), the utilization of guidelines for effective 
instructional design (Baker, et al., 2002; Gersten et al., 2009; Rosenshine, 2012), and attending 
to findings related to the effects of educational technology on academic perfonnance to guide 
decision-making (Hattie, 2009; Kulik & Kulik, 1991; Roblyer, Castine, & King, 1988) all 
informed the development of the intervention reported here. 

Purpose / Objective / Research Question / Focus of Study: 

The primary research question for this study was how successful was the classroom 
implementation of the KinderTEK mathematics intervention during the project’s Feasibility 
Testing phase? Follow up questions included: 
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1 . What was the relation between the use of an iPad-based mathematics intervention 
based on effective instructional principles and mathematics performance for 
kindergarten students? 

2. What was the nature of the relationship between usage variables and mathematics 
screening measures for all students? 

3. To what extent was student satisfaction with the KinderTEK iPad app related to usage 
and mathematics ability? 

4. To what extent were teachers satisfied with the KinderTEK intervention system? 

Setting: 

The KinderTEK feasibility study was conducted winter/spring of 2013 in a moderate 
sized school district in the Pacific Northwest. Approximately 20% of the district’s students were 
Hispanic or Latino and approximately 17% received special education services. Within the three 
participating schools, 7-27% of students were reported to be English language learners and free- 
reduced lunch eligibility ranged from 43-78% (NCES, 2013). All of the participating schools 
were Title 1 eligible and total school populations ranged from 420-530. KinderTEK was 
implemented in 10 classes; a mix of morning and afternoon classes taught by six teachers, two at 
each school. 

Participants: 

To examine the feasibility of the KinderTEK intervention for use in a tiered intervention 
system, participants for this study were a convenience sample comprised of six teachers (one 
male) with varying levels of technical expertise, and their kindergarten students. Given the 
primary development goal of supporting the learning of at-risk kindergarten students, five at-risk 
students from each class were identified as target students for the feasibility study. The target 
students were selected based on teacher recommendation, evidence of regular attendance, and 
screening data collected from student performance on a brief battery of number sense curriculum 
based measurements (Clarke et al., 2012). These 50 students were the primary focus of our 
research efforts (See Appendix B, Table 1 for complete target student demographic information). 
However, because KinderTEK has the potential to accommodate a range of learners and teachers 
expressed interest in introducing this technology to all their students, all students were invited to 
participate in the study. This increased our user-base from 50 to 213. 

Intervention: 

KinderTEK is a system of iPad applications (“apps”; Appendix B, Figure 1) providing 
mathematics intervention to struggling kindergarten students. Comprised of one app used 
individually by students and another app that allows teachers to monitor student use and progress 
in real-time, KinderTEK targets whole-number content outlined in the Common Core standards 
(CCSSO, 2010) and provides explicit instruction appropriate to each student’s learning needs. 
The intervention utilizes personalized instruction, scaffolding, feedback, and help; provides 
deliberate practice to build fluency; relies on embedded assessment; and includes teacher- 
customization, monitoring, and reporting features. KinderTEK incorporates key instructional 
design principles found to be especially effective for struggling students (Gersten et al., 2009), 
and key educational technology design principles (Mayer & Moreno, 2003) to provide quality 
instruction to students using iPads independently, customizing the learning experience for each 
student. 

Although the KinderTEK intervention has the capability to allow student choice from a 
range of appropriate activities, progress through the curriculum was constrained in this study and 
all students completed the same activities in the same order. Time spent on each activity varied, 
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however based on student accuracy and mastery of the learning objectives. Teachers were asked 
to support target students’ use of the KinderTEK intervention individually for 15 minutes per 
day, 3 days per week in addition to regular, whole class mathematics instruction. At teachers’ 
discretion and convenience, other consented students also used KinderTEK. 

Research Design: 

A mixed methods design was utilized to evaluate the feasibility and utility of the 
KinderTEK iPad mathematics intervention in authentic settings. The study reported here 
constituted the Feasibility Testing phase of an iterative cycle of technology-based intervention 
development. The primary study concerns were whether KinderTEK functioned as intended, 
whether participants enjoyed using KinderTEK, whether it could be feasibly integrated into the 
school day, and whether the data logged by the apps was accurate. For this reason, no control 
group was included. 

All student use and performance data was logged and available for analysis. Total duration of 
use was captured by log data. Because app use varied dramatically across the 213 users, progress 
through the intervention and total time of use were used to classify both target and non-target 
students into sub groups for analyses. 

Data Collection and Analysis: 

Infonnation presented here includes both qualitative and quantitative data: (a) student 
outcomes as evidenced by embedded progress monitoring data and pre/post test gains, (b) 
informal classroom observations, (c) student and teacher feedback provided through surveys and 
interviews, and (d) automatically technologically tracked KinderTEK usage data 

Student Outcomes. Student performance was monitored and logged on a daily basis to 
support the continued refinement of the intervention throughout the feasibility study. As a result, 
there is comprehensive data about individual student exposure and perfonnance within 
KinderTEK. Key summary information (e.g., number of activities attempted and completed, 
number of attempts to mastery, dosage) will be presented for both target and non-target students. 

As noted earlier, the 50 at-risk target students were the main focus of this study. The at- 
risk target students completed (a) a 0-20 number line task adapted from a 0-100 number line 
procedure (Booth & Siegler, 2008), (b) the recently published Number Sense Screener (Jordan et 
ah, 2012), (c) a collection of early mathematics curriculum based measures entitled Assessing 
Student Proficiency in Early Number Sense (ASPENS, Clarke, et ah, 2012), and (d) an 
additional oral counting Early Numeracy-Curriculum Based Measure (Chard et ah, 2005). 

Surveys and Exit Interviews. Data collectors asked students questions about their 
KinderTEK experience (e.g., likes, dislikes, frustrations; favorite activities; suggestions for 
improvement) and recorded their responses. Teachers completed an online survey with similar, 
but far more detailed questions including their own technology expertise, their use of technology 
in the classroom, and impressions of their students’ experiences with the intervention. 

Analyses. Quantitative analyses included univariate descriptive procedures and 
regression techniques to evaluate the relationship between intervention use, academic variables 
and satisfaction with KinderTEK. Qualitative analyses drew on data from teacher reports, exit 
interviews with teachers and target students, and observations of the app in use by target 
students. 

Findings / Results: 

Regression analyses conducted using usage variables (i.e. total time, activities completed, 
number of practice opportunities; Appendix B, Table 2) to predict target students’ (n = 50) gain 
scores on mathematics measures suggested that there was some evidence of a statistically 
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significant association between Kinder TEK use and changes in mathematics perfonnance in this 
brief feasibility study. Student gains in oral counting were positively related to minutes of use 
and the number of activities completed, b = .073, t = 2.015 ,p = .052, and b = 5.786, t=2.2\\,p 
< .05, respectively. Student gains on other CBM measures and pre- and post-test differences on 
summative measures were not conclusively associated with KTEK usage variables (Appendix B, 
Table 3). 

Comparative analyses conducted with the full sample (n = 213) suggested that pre-test 
ability was related to student performance within the intervention, where students with higher 
pre-test ASPENS composite scores took less time per activity, b = -. 1 80, t = -7.859, p < .00 1 , and 
got a higher proportion of practice opportunities correct, b = .001, t = 7.023,/? < .001. However, 
there was no clear statistical evidence that total progress through the KinderTEK intervention 
was related to overall post-test mathematics scores. 

Teacher and student satisfaction with the KinderTEK student app drawn from interviews 
and survey response data indicated that the majority of students and teachers were satisfied with 
KinderTEK (Appendix B, Table 4). Students were generally satisfied with KinderTEK and levels 
of satisfaction were not statistically related to academic perfonnance or usage variables. 

Teachers were least satisfied with the pacing of the content and most suggested revising the 
intervention to allow students to respond to prompts without listening to the full directions, and 
reducing the amount of verbal instructions. Teachers were moderately satisfied with the 
interface, intuitiveness, and responsiveness of the app, and were satisfied with the functionality 
of the KinderTEK teacher app (i.e., system interface and progress monitoring capabilities). 
Conclusions: 

Studies that seek to detennine the overall impact and effectiveness of an individualized, 
technologically delivered kindergarten mathematics intervention face significant challenges. 

Two of these considerations are of particular relevance to the synthesis of results from this 
feasibility study, which will be used to guide revising the intervention, and planning a pilot study 
focused on student outcomes: (1) Because each child moves through the curriculum at his own 
pace, there are challenges in grouping students into conditions and comparing mathematics 
performance. At-risk students’ intervention experiences vary greatly, thus a consideration of 
dosage (i.e., duration of use or progress through the app) may be a necessary covariate in 
analyses. (2) Although iPad-delivered interventions have many useful stand-alone, technology- 
enhanced capabilities, challenges related to technology-based implementation demands persist 
and affect teacher and student experiences when using the intervention. Separating teacher and 
student feedback about the intervention into conceptual features of KinderTEK and general 
feedback about the delivery method is an essential and complicated task. Careful survey design, 
thoughtful interview questions and clear observation protocols are all valuable tools in this 
endeavor and careful analysis of qualitative data will be essential, as well. Future research is 
recommended to evaluate the relationships between teacher feedback and technological comfort 
and experience to inform technology-based intervention implementation. 

In spite of the challenges, these results are important to the iterative process of 
curriculum development. Lessons learned throughout usability trials and the feasibility study 
reported here become valuable tools for the continued adaptation and refinement of the 
KinderTEK intervention in preparation for larger scale pilot study investigations. Technology- 
based interventions have great promise, but require thoughtful design, development, and 
continued adjustment for maximal effectiveness with kindergarten students in authentic contexts. 


Ki n derTEK_SREE201 3 fall 


4 



Appendices 


Appendix A. References 

Baker, S., Gersten, R., & Lee, D. (2002). A synthesis of empirical research on teaching 

mathematics to low-achieving students. The Elementary School Journal, 103, 51-73. 

Booth, J. L., & Siegler, R. S. (2008). Numerical magnitude representations influence arithmetic 
learning. Child Development, 79(4), 1016-1031. 

Chard, D. J., Clarke, B., Baker, S., Otterstedt, J., Braun, D., & Katz, R. (2005). Using measures 
of number sense to screen for difficulties in mathematics: Preliminary findings. 
Assessment for Effective Intervention, 30(2), 3-14. 

Clarke, B., Gersten, R., Dimino, J., & Rolfhus, E. (2012). Assessing Student Proficiency in Early 
Number Sense (ASPENS): Administrator's Handbook. Retrieved from 
http://www. soprisleaming. com/ aspens 

Clarke, B., Smolkowski, K., Baker, S. K., Fien, H., Doabler, C. T., & Chard, D. J. (2011). The 
impact of a comprehensive Tier I core kindergarten program on the achievement of 
students at risk in mathematics. The Elementary School Journal, 111(4), 561-584. 

Clements, D. (2007). Curriculum research: Toward a framework for "research-based curricula". 
Journal for Research in Mathematics Education, 35(1), 35-70. 

Collins, A., & Halverson, R. (2009). Rethinking education in the age of technology: The digital 
revolution and the schools. Teachers College Press, New York. 

Council of Chief State School Officers, National Governors Association Center for Best 

Practices. (2010). Common Core State Standards (Mathematics) . National Governors 
Association Center for Best Practices, Council of Chief State School Officers, 
Washington D.C. 

Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., . . . 
Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 
43(6), 1428-1446. 

Fuchs, D., Fuchs, L. S., & Compton, D. L. (2012). Smart RTI: A next-generation approach to 
multilevel prevention. Exceptional children, 75(3), 263-279. 

Fuchs, L. S., Fuchs, D., & Hollenbeck, K. N. (2007). Extending responsiveness to intervention to 
mathematics at first and third grades. Learning Disabilities Research & Practice, 22(1), 
13-24. 


Shanley_KinderTEK_SREE20 1 3 fall 


A-l 


Gersten, R., Chard, D. J., Jayanthi, M., Baker, S. K., Morphy, P., & Flojo, J. (2009). 

Mathematics instruction for students with learning disabilities: A meta-analysis of 
instructional components. Review of Educational Research, 79(3), 1202-1242. 

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to 
achievement. London, UK: Routledge. 

Jordan, N., Glutting, J., & Dyson, N. (2012). Number Sense Screener - K-l Research Edition. 
Paul H. Brookes Publishing Co., Inc., Baltimore, MD. 

Jordan, N. C., Glutting, J., Dyson, N., Hassinger-Das, B., & Irwin, C. (2012). Building 

kindergartners’ number sense: A randomized controlled study. Journal of Educational 
Psychology, 104, 147-160. 

Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modem mathematics computer 

games on mathematics achievement and class motivation. Computers & Education, 55, 
427-443. 

Kulik, C. L. C. & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated 
analysis. Computers in Human Behavior, 7(1-2), 75-94. doi:10. 101 6/0747- 
5632(91)90030-5 

Mayer, R., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. 
Educational psychologist, 35(1), 43-52. 

Morgan, P. L., Farkas, G., & Wu, Q. (2009). Five-year growth trajectories of kindergarten 
children with learning difficulties in mathematics. Journal of Learning Disabilities, 

42(4), 306-321. 

National Center for Educational Statistics- NCES (2013). CCD Common Core of Data. 

Retrieved from: http://nces.ed.gov/ccd/ 

Poison, D., & Morgan, C. (2010). Towards an intelligent learning system for the natural born 
cyborg .Journal of the Research Center for Educational Technology, 6(1), 185-193 

Roblyer, M. D., Castine, W. H., & King, F. J. (1988). Assessing the impact of computer-based 
instruction: A review of recent research. Computers in the Schools, 5(3-4), 41-68. 
doi: 10. 1300/J025v05n03_04 

Rosenshine, B. (2012). Principles of instruction: Research-base strategies that all teachers should 
know. American Educator, 3, 12-19. 

Slavin, R. E. & Lake, C. (2008). Effective programs in elementary mathematics: A best-evidence 
synthesis. Review of Educational Research, 78, 427-515. 


Shan ley_Ki nderTEKSREEl 01 3 fall 


A-2 



Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., ... & Yukhymenko, M. Our 

princess is in another castle: A review of trends in serious gaming for education. Review 
of Educational Research, 52(1), 61-89 


Shan ley_Ki nderTEK_SREE2 01 3 fall 


A-3 



Appendix B. Tables and Figures 


Table 1. Participants Demographics 


Target Sample 

Child Characteristics 


Child Age M (SD) 


6.23 (0.33) 

Gender N (%) 

Female 

24 (48%) 


Male 

26 (52%) 

Ethnicity N (%) 

Hispanic 

20 (40%) 


Non-Hispanic 

30 (60%) 

Race N (%) 

White 

41 (82%) 


American Indian 

4 (8%) 


African American 

0 (0%) 


Asian 

0 (0%) 


More than one race 

5 (10%) 

Free Lunch N (%) 

No 

0 (0%) 


Yes 

50 (100%) 

ELL/LEP Services Received N (%) 



No 

37 (74%) 


Yes 

13 (26%) 

SPED Services Received N (%) 



No 

39 (78%) 


Yes 

1 1 (22%) 

School Attendance M (SD) 

150.84 (18.59) 


Note. Percentages can exceed 100% as a result of rounding to 
the nearest whole number. 
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Table 2. Descriptive Report of Usage by Target and Non Target Samples 


Target Non Target 



N 

M 

SD 

Min 

Max 

N 

M 

SD 

Min 

Max 

Minutes 

50 

160.32 

79.55 

12.93 

383.53 

163 

51.25 

37.29 

0.28 

210.73 

Days 

50 

14.96 

7.05 

1 

32 

163 

5.06 

3.59 

1 

18 

Activities 

50 

3.84 

1.06 

2 

6 

163 

2.74 

0.9 

1 

5 

Practice 

Opportunities 

50 

231.12 

89.48 

31 

479 

163 

95.24 

52.44 

1 

275 

Practice Opps 
Correct 

50 

0.74 

0.08 

0.54 

0.91 

163 

0.84 

0.09 

0.52 

1 


Table 3. Dosage Response: Usage Variables and Improvements in Math Outcomes (n = 50) 


Minutes of Use Activities Attempted Practice Opportunities 

b t p b t p b t p 


ASPENS 

Oral Counting 
Numeral ID 
Magnitude Comp. 
Missing Number 
Number Line Task 
Num. Sense Brief 


07 

2.02 

.052 

01 

0.52 

.609 

01 

1.31 

.200 

01 

0.63 

.530 

01 

0.39 

.701 

01 

1.52 

.136 


5.79 

2.21 

.034 

1.31 

0.88 

.392 

.67 

0.84 

.405 

.52 

1.32 

.197 

.05 

0.14 

.893 

.80 

1.69 

.099 


08 

2.40 

.022 

02 

0.97 

.340 

01 

1.46 

.155 

01 

0.89 

.380 

01 

0.81 

.422 

01 

1.43 

.161 


Shan ley_Ki nderTEK_SREE2 01 3 fall 


B-2 



Table 4. Descriptive Report of KinderTEK Satisfaction Students (n = 50) and Teachers (n = 6) 


Students 



Really like 

Kind of like 

Kind of don’t like 

Really don’t like 

Decline to state 

Like using app 

24 

12 

2 

2 

10 

Teachers 

A great deal 

A little bit 

Not much 

Not at all 

Decline to state 

Like using app 

3 

2 

- 

- 

1 

Like content 

3 

2 

- 

- 

1 

Like functionality 

2 

2 

1 

- 

1 

Supports learners 

3 

2 

- 

- 

1 

Ease of use 

1 

4 

- 

- 

1 

Did students like 

2 

2 

1 

- 

1 

Like pacing 

- 

2 

2 

1 

1 

Like scaffolding 

- 

5 

- 

- 

1 

Like interface 

. 

5 

. 

. 

1 
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Figure 1 . KinderTEK Instructional System 
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